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Abstract 

A human- in-the-loop experiment was per- 
formed at the NASA Langley Research Center to 
study the feasibility of DAG-TM autonomous air- 
craft operations in highly constrained airspace. The 
airspace was constrained by a pair of special-use 
airspace (SUA) regions on either side of the pilot’s 
planned route. Traffic flow management (TFM) 
constraints were imposed as a required time of arri- 
val and crossing altitude at an en route fix. Key 
guidelines from the RTCA Airborne Conflict Man- 
agement (ACM) concept were applied to 
autonomous aircraft operations for this experiment. 
These concepts included the RTCA ACM defini- 
tions of distinct conflict detection and collision 
avoidance zones, and the use of a graded system of 
conflict alerts for the flight crew. 

Three studies were conducted in the course of 
the experiment. The first study investigated the 
effect of hazard proximity upon pilot ability to meet 
constraints and solve conflict situations. The second 
study investigated pilot use of the airborne tools 
when faced with an unexpected loss of separation 
(LOS). The third study explored pilot interactions in 
an over-constrained conflict situation, with and 
without priority rules dictating who should move 
first. 

Detailed results from these studies were pre- 
sented at the 5 th USA/Europe Air Traffic 
Management R&D Seminar (ATM2003). This 
overview paper focuses on the integration of the 
RTCA ACM concept into autonomous aircraft op- 
erations in highly constrained situations, and 
provides an overview of the results presented at the 
ATM2003 seminar. These results, together with 
previously reported studies, continue to support the 
feasibility of autonomous aircraft operations. 


Introduction 

NASA’s Advanced Air Transportation Tech- 
nologies project is developing a far-term concept of 
operations for Air Traffic Management (ATM). 

This concept of operations, called Distributed 
Air/Ground Traffic Management (DAG TM) [1], is 
NASA’s vehicle for researching and developing the 
RTCA Free Flight concept [2], wherein flight crews 
select their path and speed in real time while con- 
forming to restrictions established for safety and 
flow management. 

Autonomous Aircraft Operations 

DAG-TM Concept Element 5 (CE-5) [3] spe- 
cifically defines operations in the en-route and 
terminal-transition domains of flight, and it pro- 
poses the establishment of a new category of flight 
operations: autonomous flight rules (AFR). Accord- 
ing to the CE-5 concept, an AFR aircraft would 
generally operate in the same airspace as existing 
instrument flight rules (IFR) aircraft, but the AFR 
flight crew would retain a distinct set of authorities 
and responsibilities. Trained flight crews of AFR- 
equipped aircraft are given the authority to dynami- 
cally plan and execute their preferred 3D 
trajectories without coordinating with the ground- 
based air traffic service (ATS) provider. With this 
authority comes full responsibility for traffic sepa- 
ration and conformance to operational constraints; 
the ATS provider establishes these constraints in 
order to safeguard special-use airspace and manage 
traffic flows into high-demand terminal areas. Un- 
der normal operations, the ATS provider is neither 
required nor expected to intervene in AFR opera- 
tions throughout the en-route and terminal- 
transition domains. Flowever, the ATS provider 
continues to provide traditional IFR services to non- 
autonomous (“managed”) aircraft. It is anticipated 
that AFR operations would provide airspace users a 
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significant degree of flexibility to operate cost- 
effectively, and would enable the airspace system to 
accommodate a substantial increase in traffic vol- 
ume over that manageable by a ground-based 1FR 
system. This scalability would presumably result 
from minimizing the interactions between autono- 
mous aircraft operations and the ATS provider. 

The concept confines interactions between the 
ATS provider and autonomous aircraft under nomi- 
nal conditions to traffic flow management (TFM). 

In the DAG-TM concept, time-based arrival meter- 
ing will be the principal TFM tool. Using predictive 
information on arrivals and airspace status, the ATS 
provider establishes flow metering by issuing re- 
quired-time-of-arrival (RTA) clearances and 
crossing restrictions at metering fixes to AFR air- 
craft. Once these clearances and restrictions are 
received and accepted by the flight crew, the inter- 
action between the ATS provider and this 
autonomous aircraft is minimized until the aircraft 
crosses the fix. 

Flight crews of autonomous aircraft are as- 
sisted in fulfilling their AFR responsibilities by 
flight deck tools and displays integrated with the 
onboard avionics system. NASA Langley Research 
Center is developing a research prototype of this 
toolset, the Autonomous Operations Planner (AOP). 
AOP is being developed [4, 5] to integrate crew 
preferences, TFM constraints, airspace constraints 
and traffic information into the planning and execu- 
tion of efficient autonomous operations. 

RTCA Airborne Conflict Management 
Concept 

Concurrent to the development of AOP, Work- 
ing Group 1 of RTCA Special Committee 186 has 
developed an operational concept [6] for Airborne 
Conflict Management. The ACM concept includes 
three distinct functions: conflict detection (CD), 
conflict prevention (CP), and conflict resolution 
(CR). As envisaged by RTCA, the CD function will 
allow long-range detection of conflicts, providing 
flight crews the time to develop and implement an 
optimal solution to the conflict. The CP function 
will predict conflicts that could be caused by 
changes to current ownship state and intent. The CR 
function will provide guidance cues or resolution 
trajectories to solve existing conflicts. 


The RTCA ACM concept also features a pair 
of cylindrical airspace zones surrounding each air- 
craft (Figure 1). The outer zone (Conflict Detection 
Zone, or CDZ) reflects standard legal separation 
minima, while the significantly smaller inner zone 
(Collision Avoidance Zone, or CAZ) forms a tight 
“near miss” envelope around the aircraft. In the 
ACM concept, the CD function predicts penetra- 
tions of both these zones. Flight crews are alerted to 
conflicts, defined as predicted penetrations of the 
CDZ, so that they can avoid violating legal separa- 
tion minima. They are also alerted to predicted 
CAZ penetrations so that they can avert near miss 
situations. 



Figure 1. Conflict Detection And Collision 
Avoidance Zones (Not To Scale) 


Three graded CD alert levels are recommended 
by the RTCA ACM concept. A “Low Level Alert” 
indicates the presence of a conflict (i.e., predicted 
penetration of the CDZ or loss of separation) that is 
sufficiently distant to allow the flight crew the op- 
tion of developing and implementing optimized 
resolutions. A “CDZ Alert” is issued if the conflict 
is not resolved prior to a defined time threshold, to 
indicate that the conflict is proximate and that 
prompt resolution action is required. The ACM 
system also scans for possible penetrations of the 
CAZ. If a near-term CAZ penetration is predicted, a 
“CAZ Alert” is issued, indicating to the crew that 
immediate action is required to avert a possible 
collision or near miss situation. A CAZ alert is 
timed to precede the issuance of a Traffic Alert and 
Collision Avoidance System (TCAS) Resolution 
Advisory. 1 


1 In the RTCA concept, the CR function of the ACM system 
will be completely interoperable with, but functionally inde- 
pendent of, existing Airborne Collision Avoidance Systems 
such as TCAS II. 
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Experiment Objectives 

Autonomous aircraft operations in DAG-TM 
are consistent with many precepts of the RTCA 
ACM concept, allowing key guidelines from the 
RTCA concept to be integrated into the DAG-TM 
tools and procedures. The prototype AOP toolset 
has been designed to perform the three ACM func- 
tions of CD, CP and CR. AOP uses state and intent 
(FMS flight plan) information from ownship and 
traffic aircraft to perform CD, and to provide both 
strategic and tactical CR to solve intent-based and 
state-based conflicts respectively. It also provides 
CP guidance using the state-based Predictive Air- 
borne Separation Assurance System (P-ASAS) 
concept pioneered by the NLR [7]. Introducing the 
concepts of CDZ and CAZ into AOP enhances its 
capability to support all ACM functions. In addi- 
tion, introducing into AOP the graded system of 
alerts outlined by RTCA provides a simple and 
intuitive framework for crew alerting to conflicts. 
The study presented here tested the integration of 
these key features of the RTCA ACM concept into 
AOP, and assessed pilot use of these ACM func- 
tions, with the overall objective of exploring the 
feasibility of autonomous aircraft operations. 

The fundamental challenge of autonomous air- 
craft operations is ensuring that flight crews of 
autonomous aircraft, upon receiving and accepting 
TFM constraints, are able to plan and execute effi- 
cient conflict-free trajectories that meet all TFM 
constraints within established tolerances. In current- 
day operations, air traffic controllers perform simi- 
lar tasks for all aircraft within their sectors. Flight 
crews of autonomous aircraft, however, have to 
deal with only those conflicts that involve their own 
aircraft; that is, they need to solve only a subset of 
all conflicts in the airspace, thereby reducing con- 
flict management workload to a linear function. 
Further, the introduction of ADS-B provides long- 
range surveillance, and hence long look-ahead peri- 
ods are available for conflict detection. With fewer 
aircraft to consider for separation assurance, greater 
time and flexibility to solve conflicts, and the abil- 
ity to monitor developing situations more closely, it 
is hypothesized that flight crews of autonomous 
aircraft may indeed be able to readily manage 
autonomous flight through nominal environments, 
and do so in significant traffic densities. 


A variety of research studies support this hy- 
pothesis. Numerous approaches to providing both 
unconstrained [7,8] and TFM-constrained [9,10] 
airborne separation assurance have been developed 
and successfully tested in nominal conditions over 
the past several years. In an earlier piloted simula- 
tion using the AOP prototype, the authors studied 
tactical and strategic modes for autonomous aircraft 
operations with TFM constraints, and found that 
both operational modes are consistent with, and 
support feasibility of, autonomous aircraft opera- 
tions at up to three times current traffic density [10]. 

However, in order to fully explore autonomous 
aircraft operations and establish the limits of feasi- 
bility of the underlying concept, it is necessary to 
study more challenging situations involving highly 
constrained and off-nominal conflict scenarios. 
Investigations of such situations provide better in- 
sight into concept feasibility and concept 
robustness, while providing guidelines for the con- 
tinued development of airside ACM tools and 
groundside approaches to TFM that minimize air- 
ground interactions in autonomous aircraft opera- 
tions. 

A highly constrained situation relevant to con- 
cept feasibility is one where airspace hazards, 
traffic flows and TFM constraints all co-exist to 
pose a simultaneous challenge to autonomous air- 
craft operations. Several research questions can be 
addressed in these challenging situations. For ex- 
ample, what is the effect of the proximity of hazards 
and traffic on the ability of autonomous aircraft to 
successfully satisfy all constraints? Can conflicts in 
these situations be resolved without additional 
air/ground interaction? Off-nominal situations are 
also of interest, including cases where the full 
nominal sequence of alerts is not presented to the 
pilot due to ADS-B range limitations, improper 
maneuvering by another pilot, emergency maneu- 
vering by another pilot, etc. How would pilots 
respond to the resulting near-term “pop-up” con- 
flicts? Another off-nominal situation of import 
probes the interdependence of conflict avoidance 
and TFM constraints. What if it were literally im- 
possible for all constraints to be simultaneously met 
- will safety be compromised if pilots are unaware 
of the over-constrained nature of the situation? 

The experiment reported herein addressed 
these issues in a human-in-the-loop simulation of 
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autonomous aircraft operations. The exploratory 
study had three specific objectives: (a) to determine 
the effect of varying hazard proximity upon pilot 
ability to maintain separation assurance and adhere 
to TFM constraints; (b) to investigate pilot use of 
AOP in near-term, pop-up conflicts; and (c) to de- 
termine the nature of pilot interactions in an over- 
constrained conflict situation. Investigations into 
objectives (a) and (b) above included a preliminary 
investigation of the effect of lateral separation crite- 
ria upon pilot use of the prototype ACM system. 
Numerous other factors would have to be explored 
to fully evaluate whether separation criteria could 
be modified, and making a case for adopting a 
smaller separation criterion is well outside the 
scope of this study. Investigations into objective (c) 
of the study also probed the need for and effective- 
ness of priority flight rules in preserving traffic 
separation in the over-constrained conflict situation. 
For this portion of the study, priority rules were 
integrated into the alerting scheme that was imple- 
mented in AOP. In total, the three objectives 
allowed a probing of the limits of concept feasibil- 
ity and an assessment of the effectiveness and pilot 
interaction with a prototype decision support tool 
with capabilities based on the RTCA ACM concept. 


Experimental Approach 

The experiment was performed in the Air Traf- 
fic Operations Laboratory (ATOL) at the NASA 
Langley Research Center. The ATOL is a distrib- 
uted simulation environment where pilots of 
multiple simulated aircraft can interact in pre- 
planned or dynamically developing scenarios. As 
described earlier, features of the RTC A ACM con- 
cept were integrated into ATOL aircraft simulators 
for the experiment. 

As part of this integration, AOP (as described 
in [4]) was modified in several ways. CD was ex- 
panded to perform state-based CD on the CAZ. The 
alerting scheme by which AOP alerts the crew to 
conflicts was modified to use the graded alert levels 
recommended by RTCA. The time-lines for the 
alerts were selected in accordance with the ranges 
recommended by RTCA (Figure 2). 

In addition, the CP system in AOP (based on 
NLR’s P-ASAS concept) was extended to provide 
preventive information on CAZ avoidance. While 
not a specific recommendation in the RTCA ACM 


concept, this capability was tested as a logical ex- 
tension to the nominal CP system. The resulting CP 
system provides three levels of CP guidance. The 
time-horizons for these three levels of CP were set 
to 1 minute (red band, indicating collision preven- 
tion), 2 minutes (amber band, indicating “do not 
fly” zone), and 5 minutes (cyan band, indicating 
“proceed with caution”). 


RTCA Guidelines for Alert Thresholds: 

CAZ Alert: ~ 1 minute prior to CAZ penetration 

CDZ Alert: between 2 and 5 minutes prior to CDZ penetration 

Low Level Alert: between initial detection of conflict and CDZ Alert 


AOP implementation: 



10 


5 2 LOS 

Minutes to LOS 


Figure 2. Implementation Of RTCA Guidelines 


Where the ACM concept is silent on the issue 
of what traffic may be of interest to the crew, the 
ATOL implementation assigned an additional 
“point-out” alert level for “possible threat” aircraft. 
These were traffic aircraft that were not in conflict 
but could still be of interest to the crew, either due 
to being proximate, or being within easy range of a 
conflict. Traffic aircraft not qualifying for one of 
these four alerts were not displayed. 



Figure 3. Multiple Layers Of Protection 

With the integration of RTCA guidelines into 
AOP, multiple layers of protection are now pro- 
vided for separation assurance in autonomous 
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aircraft operations (Figure 3). Simulation studies of 
Free Flight in en route airspace [11] have suggested 
that the likelihood of conflicts is rather low (i.e., big 
sky), and if conflicts do arise, CE-5 introduces re- 
dundancy in conflict management by distributing 
the responsibility for separation assurance. The 
ACM toolset then provides flight crews with sev- 
eral additional layers of protection before they enter 
the realm of visual separation assurance and AC AS: 
CP tools help prevent the creation of new conflicts; 
Low Level Alerts based on intent and state informa- 
tion can detect conflicts early and reduce missed 
alerts; CDZ Alerts introduce a measure of urgency 
by prompting pilots to take timely avoidance ac- 
tions; and CAZ Alerts provide pilots with additional 
time to react to potential collision situations. 

Experiment Design 

Sixteen currently or recently active airline pi- 
lots participated in the experiment, which was 
conducted over a 2-week period in the Fall of 2002. 
Only the autonomous aircraft operations aspect of 
CE-5 was represented in the simulation; the ATS 
provider and piloted managed aircraft were not 
simulated, nor were they needed to achieve the 
experimental objectives. The pilots were spread 
among four groups, which flew in the simulation 
one group at a time. Efforts were taken to disguise 
the fact that pairs of them would occasionally inter- 
act through traffic conflicts. The majority of 
conflicts were scripted with automated aircraft. 

The pilots received comprehensive training on 
the procedures, alerting levels, and conflict man- 
agement tools through printed materials, classroom 
briefings, and hands-on practice with one-on-one 
instruction before they flew the planned test scenar- 
ios. Additionally, the first test scenario after 
training was in fact a buffer scenario, unbeknownst 
to the pilots, to verify the required pilot proficiency 
level had been reached. Including this buffer sce- 
nario, each pilot flew a total of 10 scenarios, 
counterbalanced for order, each based on the same 
basic scenario design described below. 

Experiment Scenarios 

The basic experimental scenario consisted of 
an en-route trajectory constrained by special-use 
airspace. The sizes and shapes of these SUAs were 


realistic, motivated by a current airspace configura- 
tion in the area of Reno, Nevada, USA. An 
autonomous aircraft, controlled in the experiment 
by a single human pilot, was established on a flight 
plan between the SUAs [Figure 4(a)], In addition to 
the subject-piloted aircraft, the airspace was popu- 
lated with other aircraft traveling in both directions 
through the corridor at altitudes above, below, and 
equal to that of the subject pilot’s aircraft. This 
traffic provided a realistic and distracting environ- 
ment for the subject pilot. Note that a common 
structured route through the corridor was not used 
for any autonomous aircraft in the scenario. 

To meet hypothetical TFM needs (details not 
included or required in the simulation), the flight 
plan of the aircraft included a waypoint with a re- 
quired time of arrival (RTA) and a “cross at” 
altitude restriction, both of which were supposedly 
assigned by the ATS provider. These constraints 
were pre-entered in the Flight Management System 
(FMS) flight plan at the outset of the simulation, 
which began approximately 200 miles (25 minutes) 
before the RTA waypoint. These constraints pro- 
vide quantifiable metrics against which the 
effectiveness of the cockpit decision tools and dis- 
plays could be evaluated, and also provide mission 
goals for the subject pilots. Crossing the RTA way- 
point within ±30 seconds of the assigned time, 
within ±500 feet of the assigned altitude, and within 
±2.5 NM laterally was defined to be acceptable 
conformance to the TFM constraints. The pilots 
were briefed on these tolerances. 

Subject pilots were instructed to maintain traf- 
fic separation as a top priority, and to achieve the 
assigned waypoint constraints as a second priority. 
If the pilot determined that any of the assigned con- 
straints could not be met for any reason, the 
instructions were to “notify” the ATS provider as 
early as possible, using three labeled buttons pro- 
vided in the flight simulator for this purpose. Pilots 
were not constrained to follow the initial flight plan. 
Rather, they were advised to make their own best 
judgments regarding the conduct of the flight. The 
pilots were free to choose the lateral and vertical 
path that they felt best met their objectives. Flights 
were not constrained to the hemispherical altitude 
flight levels in accordance with the CE-5 concept. 
Therefore, it was possible for subject pilots to en- 
counter unplanned conflicts and flight situations. 
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This basic scenario was modified as required 
to investigate the three main research objectives. 
Research issues common to all scenarios included 
safety of flight operations through the corridor, 
acceptability and usability of the cockpit tools, and 
the pilot use of combined strategic and tactical reso- 
lution tools. 

Hazard Proximity Scenarios 

The primary research issues for this part of the 
study were the interactive effects of required lateral 
separation and constrained airspace on pilot ability 
to maintain separation and meet assigned con- 
straints. The independent variables studied were the 
width of the SUA “corridor” and the lateral dimen- 
sions of the CDZ surrounding each aircraft. In 
addition, unbeknownst to the subject pilot, there 
was one planned conflict, with separation predicted 
to be lost approximately halfway through the corri- 
dor. 

The basic scenario was studied with two corri- 
dor width conditions: a narrow corridor and a wide 
corridor [Figure 4(b)]. Each subject pilot flew both 
conditions as part of the within-subjects experiment 
design. The narrow corridor had a minimum width 
of approximately 33 nm. The wide corridor had a 
minimum width of approximately 65 nm. For data 
collection flights through the wide corridor, addi- 
tional traffic aircraft were added to the simulation to 
maintain the aircraft density and, hence, the diffi- 
culty of the conflict avoidance task. 

The second independent variable studied was 
the lateral dimension of the CDZ. Again, two condi- 
tions were studied: a CDZ with a 3 NM radius and a 
CDZ with a 5 NM radius. Each data collection 


flight involved only one of these conditions. Subject 
pilots were briefed on the CDZ size in effect prior 
to each flight, and the AOP algorithms were ad- 
justed to the appropriate size of the CDZ. In all 
cases, the vertical dimension of the CDZ was ±1000 
feet. 

Each pilot flew all combinations of the 2x2 
matrix of corridor width and separation criteria, for 
a total of four hazard proximity runs per pilot. Data 
from five runs was lost to simulation faults, result- 
ing in a total of 59 valid hazard proximity runs for 
data analysis. 

Pop-up Conflict Scenarios 

This portion of the study was intended to 
evaluate the effectiveness of the prototype ACM 
tools in near-term conflict situations that could be 
caused by, for example, an aircraft that is maneu- 
vering improperly in response to a conflict alert, or 
an aircraft forced to descend rapidly due to some 
emergency. In addition, the study compared the 
effect of 3 and 5 NM lateral separation standards 
(with 1 000 ft vertical separation) on pilot ability to 
safely resolve near-term traffic conflicts and regain 
unexpectedly lost separation. 

The basic scenario of Figure 4 (a) was used for 
this portion of the study. In order to simulate an 
unexpected near-term conflict, a scripted intruder 
aircraft was maintained on a track parallel and co- 
altitude to the subject piloted aircraft until a pre- 
determined time in the scenario. However, this in- 
truder was “hidden” from the subject pilot by 
suppressing its ADS-B broadcast. This simulation 
procedure was necessary to create the unexpected 
pop-up event. 



(a) Basic Scenario (b) Hazard Proximity Scenario (c) Over-Constrained Scenario 


Figure 4. Experiment Scenarios 
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At the appropriate time, generally as the sub- 
ject aircraft was emerging from the SUA corridor, 
the simulation manager turned the intruder toward 
the subject aircraft and activated the intruder’s 
ADS-B broadcast. As a result, the intruder would 
“pop-up” about 6 NM away from the subject air- 
craft, on a conflicting heading and at the same 
altitude. Due to the proximity of the intruder at first 
appearance, separation would almost certainly be 
lost and immediate maneuvering would be required 
to regain required separation. The same conflict 
geometry (location, approach angle, and time to 
closest approach) was used for both 3 NM and 5 
NM separation zone cases. This allowed a compari- 
son of risk incurred and highlighting of any 
variation in pilot performance due to an aircraft 
being inside or outside the CDZ. 

Each pilot flew two runs with pop-up conflicts. 
Of the resulting 32 runs, four runs were marred by a 
simulation failure late in the run. Data on the pop- 
up conflict and consequent pilot responses were not 
contaminated in these four runs, and were included 
in subsequent data analysis. However, the constraint 
conformance data were not useable. 

Over-Constrained Conflict Scenarios 

The scenarios used to investigate this objective 
were identical to the basic scenario of Figure 4(a), 
except that the flight plans of two subject-piloted 
aircraft intersected at the constraint waypoint, 
which could be considered a metering fix 2 . The 
over-constrained conflict was created by issuing 
identical crossing constraints (RTA and altitude) at 
that waypoint to both aircraft [Figure 4(c)]. Of 
course, a significant failure at several levels of the 
NAS must have occurred to permit this situation to 
arise at all; nevertheless, the objective of this study 
was specifically to determine how the airborne side 
might react to such potentially hazardous situations 
in an effort to test the robustness of distributed 
autonomous aircraft operations. 

Efforts were taken to ensure that both pilots 
were not aware a priori of the nature of this conflict, 
nor that they would be interacting with another 
subject pilot in this scenario. Each pilot pair en- 
countered this scenario three times. For two of these 


runs priority rules (governing which aircraft should 
move first) were in effect. The two runs were con- 
ducted so that each pilot had a turn at higher and 
lower priority, to counterbalance the test matrix. 
Priority was not actively indicated to the pilots; 
rather a passive technique was used in that the alert 
level (i.e. point out. Low Level Alert, CDZ Alert) 
was staggered in time to favor the higher priority 
aircraft, until an established time threshold was 
reached when both aircraft received CDZ Alerts 
(Figure 5). The third run was performed with no 
priority rules; that is, both pilots received alerts at 
the same time (as was indicated in Figure 2). Com- 
parison of the data from these runs provides insight 
into the effectiveness of implicit coordination 
(through priority rules) in resolving this challenging 
situation. 


□ Point-out 

□ Low Level Alert 



10 52 LOS 

Minutes to LOS 


Figure 5. Implementation Of Priority Through 
Staggered Alerting Thresholds 

Of the 48 data runs conducted to investigate 
these issues, four runs were lost to simulation faults. 
The remaining 44 runs were analyzed in pairs (since 
pilots interacted in pairs in the course of this sce- 
nario). Of these 22 pairs of runs, 10 were conducted 
without priority rules in effect, and 12 had priority 
rules in effect. 

Results And Discussion 

Results from these studies were previously re- 
ported at the 5 th USA/Europe ATM R&D Seminar 
(ATM2003) [12-14], This overview paper focuses 
on the integration of the RTCA ACM concept into 
autonomous aircraft operations in highly con- 
strained situations, and provides an overview of the 
results presented at the ATM2003 seminar. 


2 Descents were not included in the scenario because proper 
FMS descent capability in the simulation had not yet been 
completed. 
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Safety Of Flight Operations 

A fundamental indication of whether safety 
was compromised in the course of experiment sce- 
narios is the occurrence of loss of separation events. 
Except for the scripted pop-up conflicts, pilots were 
expected to receive sufficient alerts ahead of time to 
be able to use the ACM tools and solve the conflict. 
However, a few LOS occurrences were observed in 
the course of the hazard proximity and over- 
constrained runs. 

Not counting simulation faults, pilots lost sepa- 
ration twice in the hazard proximity runs. One LOS 
situation resulted from a subject-pilot initiating a 
climb without proper consultation of CP informa- 
tion presented on the Primary Plight Display (which 
warned him against the climb). The other LOS 
situation occurred when a pilot initiated a vertical 
maneuver without making certain that required 
lateral separation existed with traffic aircraft 3 . 

In the over-constrained runs, there was one 
LOS occurrence not due to a simulation fault. The 
two subject-piloted aircraft lost separation close to 
the RTA waypoint when one of the pilots climbed 
prematurely after passing the other aircraft, without 
actively ensuring that lateral separation with the 
other aircraft existed, again ignoring vertical CP 
information. 

Closer examination of these three LOS situa- 
tions indicates that none of them represented a 
significant safety risk; rather, they were in the na- 
ture of minor infractions. These LOS situations, 
however, do point to areas of potential improve- 
ment in pilot training and the display of CP 
information. The low incidence of LOS situations 
overall indicate that pilots were largely successful 
in maintaining safe flight in the course of these 
scenarios. 

Overall Constraint Conformance 

As was stated earlier, a major challenge in 
autonomous aircraft operations is the ability of 
flight crews to successfully plan and execute con- 
flict-free paths that meet all TPM constraints. Data 
collected in the course of the hazard proximity and 


3 The pilot failed to use a display feature that would have dis- 
played a circle on the Nav Display indicating the traffic’s 
separation zone. 


pop-up conflict scenarios indicate that pilots were 
largely successful in performing these tasks. The 
over-constrained conflict scenarios, as stated ear- 
lier, were designed such that both pilots could not 
simultaneously meet the TFM constraints. Figure 6 
indicates the observed deviations from assigned 
constraints in the course of all runs employing the 
first two scenario types. 


■ Hazard Proximity □ Pop-up 



Distance from Fix (NM) 


Figure 6. Deviations From Assigned TFM 
Constraints 

It is clear that an overwhelming majority of pi- 
lots successfully met all TFM constraints in these 
scenarios. Further, a majority of pilots met their 
constraints well within permitted tolerances. This 
behavior is true even in the runs featuring pop-up 
conflicts, where sudden conflicts appeared about 
midway through the run. These results support fea- 
sibility of autonomous operations even under highly 
constrained and off-nominal situations. 

Pilot opinions on alerting scheme 

Following the end of each data run, pilots were 
asked to fill out a questionnaire that included the 
following question: “How intuitive was the conflict 
alerting system? An intuitive alerting system would 
provide clear information about which aircraft you 




were in conflict with and the urgency of the con- 
flict. ” The ordinal rating scale ranged from “not at 
all intuitive” (1) through “neutral” (4) to “very in- 
tuitive” (7). Pilot opinions across all run types are 
presented in Figure 7. 

The data indicate that mean responses were 
above neutral for all run types: 5.85 for hazard 
proximity runs, 4.96 for pop-up conflict runs, and 
5.52 for over-constrained runs. Of the 130 runs for 
which data is presented here, pilots in 1 1 0 runs 
(85%) rated the alerting system to be above neutral, 
compared to just 1 1 runs (8.5%) where pilots rated 
the system to be below neutral. These results indi- 
cate that pilots found the implementation of the 
RTCA ACM alerting concept to be useful and intui- 
tive in performing autonomous aircraft operations. 



Figure 7. Pilot Opinions On Intuitiveness Of 
Alerting 


Hazard Proximity 

The objective of this part of the experiment 
was to investigate the effect of hazard and traffic 
proximity on the ability of autonomous aircraft to 
successfully satisfy all constraints. Overall results 
[12] from this part of the experiment indicate that 
airborne separation is feasible even under the tight 
airspace and arrival constraints, and that efficien- 
cies are gained with a reduction of the lateral 
separation standard. The graded system of ACM 
alerts was effective in enabling both strategic and 
tactical management of conflicts within the opera- 
tional constraints defined by the scenario. 

Stability of the overall traffic situation can be 
augmented if conflict resolutions have minimal 


impact on other aircraft in the vicinity. In all scenar- 
ios used in this experiment, special care was taken 
to ensure that there would be no other conflicts 
except the one planned along the original flight 
plan. Therefore, any additional conflicts that oc- 
curred during a scenario were due to the pilot 
deviating from the flight plan to avoid the planned 
conflict. These “second generation” conflicts 
constitute a ripple effect of the original conflict 
resolution. Data collected in the hazard proximity 
scenarios were analyzed for occurrences of second- 
generation conflicts. 

Figure 8 shows the distribution of second- 
generation conflicts across pilot strategies to resolve 
the conflict in these scenarios. Here “strategic” 
resolutions indicate the pilot remained in FMS 
guidance throughout the resolution. “Modified stra- 
tegic” resolutions are those where the pilot initially 
accepted a strategic resolution, but chose to modify 
it with a subsequent maneuver. “Tactical” resolu- 
tions indicate maneuvers using the Flight Control 
Panel and not the FMS. It is seen that pure-strategic 
resolutions caused no additional conflicts. This is to 
be expected, since strategic resolutions offered by 
AOP are designed to be conflict free for the next 20 
minutes, which typically approached the length of 
the remaining scenario. However, tactical resolu- 
tions do not take into account other aircraft besides 
the conflicting aircraft, and the pilot was responsi- 
ble for avoiding future conflicts through judicious 
use of the CP system. Further, for the pilots using 
the modified strategic resolution method, all second 
generation conflicts occurred after the tactical 
modification. These findings suggest that enhancing 
the tactical resolution system to take into account 
other aircraft would reduce conflict proliferation. 



Figure 8. Second-Generation Conflicts 
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Further analysis [12] of the data indicated that 
both strategic and tactical maneuver choices trig- 
gered an alert for second-generation conflicts. Thi s 
analysis points to the need to provide CD on all 
provisional “what-if ’ trajectories; modifications to 
AOP are in progress to accomplish this function. It 
also motivates closer examination of what look- 
ahead times are appropriate for conflict prolifera- 
tion avoidance. 

Pop-up Conflicts 

The motivation for this part of the experiment 
was to assess pilot use of the ACM tools in re- 
sponding to a near-term conflict alert, and to 
investigate the effect of reducing the lateral separa- 
tion standard on pilot task of regaining separation. 

Analysis of the data collected during these 
scenarios used a non-dimensional parameter, epsi- 
lon [15], as an indicator of threat severity. 
Dimensional quantities such as distance at closest 
point of approach do not reflect the unequal lateral 
and vertical extent of current-day separation stan- 
dards. Epsilon normalizes the lateral and vertical 
components of distance between aircraft to the lat- 
eral and vertical extent of the CDZ. In order to 
compare results from 3 and 5 NM scenarios, all 
results using epsilon were calculated relative to a 5 
NM standard. Analysis of the data [13] determined 
that compliance with the tactical guidance provided 
by the ACM tools reduced severity of the threat as 
indicated by higher epsilon values. 
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Note: Error bars represent 1 standard error of the mean. 


Figure 9. Risk Mitigation Using ACM Tools 


Analysis also focused on risk mitigation by pi- 
lot actions following the conflict alert. This analysis 
compared the epsilon value predicted at first alert 
appearance with the minimum epsilon actually 
achieved by the pilots’ maneuvers. Large differ- 
ences between these quantities indicate better risk 
mitigation by the pilot. Figure 9 indicates the find- 
ings of this analysis. It is seen that compliance with 
ACM tool guidance provided better risk mitigation 
for both 3 and 5 NM separation standards. How- 
ever, the change in the lateral standard caused no 
significant difference in risk mitigation. 

Over-Constrained Conflicts 

The objective of this part of the experiment 
was to study pilot interactions in an over- 
constrained conflict situation, and to evaluate the 
usefulness of implicit coordination through priority 
rules in the resolution to this conflict. Results from 
the study [14] indicated that conflicts were safely 
resolved, both with and without priority rules. The 
following discussion focuses on the effect of prior- 
ity rules upon constraint conformance among the 
pilot pairs. 

The constraint conformance of the subject air- 
craft across all over-constrained data runs is 
presented in Figure 10. Three comparisons are pre- 
sented: (1) all subject-aircraft with and without 
priority flight rules in effect; (2) subject-aircraft 
approaching from the right-hand (RH) and left-hand 
(LH) directions when priority flight rules were not 
in effect; and (3) subject-aircraft with and without 
priority when flight rules were in effect. 

These results (1) indicate that approximately 
one-third of the aircraft met all of the waypoint 
constraints (time, position, and altitude) within 
acceptable tolerances, regardless of whether the 
priority flight rule system was used. When priority 
flight rules were in effect (3), aircraft that met all 
constraints were always the higher priority aircraft, 
indicating that the use of priority flight rules in- 
creases predictability regarding which aircraft in a 
pair is likely to prevail. This data also indicates the 
implementation of priority flight rules through stag- 
gered alerting thresholds was completely effective. 
Interestingly, when priority flight rules were not 
used (2), the RH aircraft in a conflict pair met their 
constraints about 2.5 times as frequently as the LH 
aircraft, suggesting that pilots may have subcon- 
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sciously applied previously-leamed standards for 
priority (such as VFR priority rules). 



Figure 10. Constraint Conformance In Over- 
Constrained Scenarios 

Closer examination of these data brings into 
focus some important issues regarding broadcast 
intent. With priority flight rules in effect (3), in only 
two-thirds of the cases did one aircraft of the pair 
(the higher-priority aircraft) meet all constraints. 

The expectation was that all pilots with priority 
should have met all constraints. An investigation of 
the conflict alerting revealed that, in three of the 
four cases where the higher-priority (HP) aircraft 
did not meet all constraints, the HP aircraft received 
a conflict alert after the lower-priority (LP) aircraft 
had already implemented a strategic resolution. The 
lateral path-stretch in these strategic resolutions 
should have created an arrival delay that would 
have eliminated the conflict at the RTA waypoint. 
The conflict was still registered, however, because 
the LP aircraft were broadcasting their required 
time of arrival at the waypoint, rather than their 
estimated time of arrival. These data illustrate a 
hazard of broadcasting unachievable flight plan 
constraints, essentially false intent, in place of true 
trajectory predictions. Neighboring aircraft may use 
this false information to make maneuver decisions 


that, at a minimum, disrupt flight efficiency but 
may also lead to new conflicts. Broadcasting the 
commanded trajectory as the intent message, i.e., 
the four-dimensional path the autoflight system will 
actually command (assuming no further pilot in- 
puts), would reduce or eliminate the hazards 
associated with disseminating false information. In 
fact, a recent update to the ADS-B system perform- 
ance standards recommends the broadcast of 
commanded trajectory information [16]. 

Conclusions And Future Work 

The feasibility of autonomous aircraft opera- 
tions in highly constrained airspace was studied 
using a multi-piloted simulation. Pilots were tasked 
with negotiating an airspace corridor in the pres- 
ence of traffic, and were required to meet TFM 
constraints at a waypoint beyond the corridor. As 
part of the experiment, RTCA guidelines for Air- 
borne Conflict Management were integrated into a 
prototype flight deck tool to assist the pilot in per- 
forming CD, CP and CR functions. The experiment 
yielded the following conclusions: 

• AFR operations were feasible under the tight 
airspace and arrival constraints featured in 
this experiment. Pilots found the ACM alert- 
ing to be intuitive, even in scenarios 
featuring pop-up conflicts. The graded sys- 
tem of alerts was effective in enabling 
strategic and tactical conflict management. 
Safety of operations was not compromised 
by the combination of constraints. 

• Reducing the lateral separation standard had 
no adverse effect on pilot ability to meet 
TFM constraints, and reduced the incidence 
of second-generation conflicts. In near-term 
conflicts, pilot ability to reduce threat prox- 
imity and mitigate risk appeared to depend 
more on compliance with the tactical resolu- 
tion guidance than on CDZ size. 

• The use of priority rules improved predict- 
ability of constraint conformance but was 
not critical for ensuring separation in over- 
constrained scenarios. The implementation 
of priority flight rules through staggered 
alerting thresholds was completely effective. 

• CD, CR and CP tools must be enhanced to 
assist pilots in all possible guidance modes. 
Broadcasting the commanded trajectory 


11 



rather than an FMS flight plan would reduce 
unnecessary maneuvering and adverse air- 
craft interactions resulting from false 
conflict alerts for the receiving aircraft. 

These features are being incorporated in the 
continued development of AOP. 

Future Work 

The next step in investigating the feasibility of 
autonomous aircraft operations is to examine the 
interactions between autonomous aircraft and the 
Air Traffic Service provider, and the effect of 
autonomous aircraft operations on managed aircraft 
operations. An integrated air/ground simulation 
experiment is currently being designed by NASA 
Langley and Ames Research Centers to address 
these issues. The experiment will involve subject 
controllers and subject pilots, and will assess the 
feasibility of autonomous aircraft operations in the 
arrival flow to a terminal-entry metering fix. 
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